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TITLE: Video editing apparatus, video editing method, and data storage medium for a 
video editing program 

Abstract Text (1) : 

Picture quality loss is minimized and buffer underflow and overflow are prevented 
when extracting and recombining a plurality of consecutive scenes from an MPEG 
stream. The smallest number of pictures (equivalent at most to 1 second) that must 
be re-encoded so that the scene can be independently reproduced is detected to 
identify the re-encoding target picture. Then the picture type (I, P, B) after the 
pictures are re-encoded is determined. Buffer fullness is calculated, and the code 
allocation to the re-encoding range is calculated from buffer fullness and target 
picture count. The code allocation to each target picture is then calculated, and 
the target pictures are re-encoded and connected. 

Brief Summary Text (16) : 

and the code size is conversely 

Brief Summary Text (49) : 

More specifically, data extractor 12 sends the frame composition of each scene 
(that is, the picture types in the scene) to the control point determining means 
13. Using this frame composition information sent from data extractor 12, control 
point determining means 13 determines whether ther e is a frame at tv^p ^ rjinning r> r 
end of the scene that must be re-encoded in order to sustain the picture content at 
the beginning and end of the scene, and defines any such frame as a variable bit 
rate frame. That is, if a picture referenced to code a particular frame in the 
display order is not included in the group of frames constituting the scene, that 
particular frame is designated a variable bit rate frame to be re-encoded with a 
different code size. If there is a plurality of consecutive variable bit rate 
frames, these frames are treated as a variable bit rate frame group. 

Brief Summary Text (55) : 

Operation of the bit rate controller 14 is described next. The bi t rate controll er 
14 re-encodes each of the variable bit rate frames to I-pictures based on t he 
target code size allocated to each variable bit rate frame by code size calculator 
15. It is also det ected at th i s time whether c o di n q_Lo^a.n^I--^-e-few?e— i^— pos-s-iirl e 
using 'the target code size. I"f not, a number of P-pictures with a difference of 0 
is inserted, and the increased code size is added to the target code size of the" 
variable bit rate frame. After thus re-encoding the variable bit rate frames, scene 
data is relinked to produce a new bitstream. 

Brief Summary Text (59) : 

Buffer fullness is analyzed for every picture in a scene in order to calculate the 
code size of the re-encoded pictures. Depending on the scene length, this may 
require processing a large amount of data. 

Brief Summary Text (63) : 

All re-encoded pictures are I-pictures. I-pictures consume a large amount of code. 
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Brief Summary Text (65): 

Code size is evenly allocated to plural re-encoded pictures at the point the scenes 
are edited. If there are many re-encoded pictures and a small amount of code is 
allocated, picture degradation propagates and the quality of the entire video 
sequence drops. 

Brief Summary Text (70) : 

To resolve the first and second tasks, and problems 1 and 2 above, a video editor 
according to the invention has a scene information input means for inputting scene 
information, where a scene is a plurality of consecutive frames extracted from an 
edit stream; a re-encoding target picture selector for selecting as target pictures 
for re-encoding the smallest number of pictures that must be re-encoded for the 
scene to be independently reproducible; a stream structure data generator for 
generating structure information for the stream range containing a target picture 
and an intra-coded picture referenced for coding the target picture; a buffer 
fullness calculating means for calculating, from stream structure data, buffer 
fullness, or buffer occupancy, at a target picture boundary to a recycled picture 
not requiring re-encoding; a re-encoding range code allocation calculator for 
calculating a code allocation to a re-encoding range based on buffer fullness and 
target picture count, said re-encoding range being one or a plurality of target 
pictures near an edit point between scenes; a re-encoding target picture target 
code size calculating means for calculating a code allocation to each target 
picture based on the re-encoding range code allocation, target picture count, and 
picture type after re-encoding ; and a scene linking means for connecting scenes and 
producing a new stream. 

Brief Summary Text (71) : 

To resolve problem 3 above, the video editor of the invention further preferably 
has a re-encoding target picture type determining means for deciding, from stream 
structure data, a target picture picture type after re-encoding ; and a picture re- 
encoding means for target picture re-encoding based on target picture target code 
size and picture type after re-encoding . 

Brief Summary Text (72) : 

To resolve problem 4 above, the video editor of the invention further preferably 
has a code allocation verifying means for verifying whether the code allocation is 
appropriate based on the re-encoding range code allocation and target picture 
picture type after re-encoding; and a re-encoding range expanding means for 
extending the re-encoding range if the code allocation is not appropriate. 

Brief Summary Text (73) : 

Yet further preferably to resolve problem 4 above, the video editor of the 
invention preferably has a re-encoding target picture importance calculating means 
for calculating an importance rating for each target picture in the re-encoding 
range . 

Drawing Description Text (9) : 

FIG. 8 is a graph showing the change in buffer fullness before and after the re- 
encoding range extension process at the beginning of a scene; 

Drawing Description Text (11) : 

FIG. 10 is a graph showing the change in buffer fullness before and after the re- 
encoding range extension process at the end of a scene; 

Detailed Description Text (6) : 

Re-encoded picture: One or a plurality of encoded pictures at the beginning of the 

extracted scene, or one or a plurality of encoded pictures at the end of the 

extracted scene, that is decoded and then re-encoded . Also referred to as "target 
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picture." These pictures are indicated by the solid black blocks in FIG. 12. 
Detailed Description Text (10) : 

De pending upon the scene extracted from the bitstream, there will be pictures (such 
as a B-picture) at or near tne peginnincf of the extracte d scene that are encode d 
with correlation to a pict ure outside the ex tracted scene. Such pictures cannot be 
r eproduced using only the^picture inf ormation^g ontained in the extracted sce ne, and 
it is therefore necessary to re-encode either just the first extracted picture or 
this first picture and some adjacent pictures. The re-encoding process of the 
present invention is described next below with reference to FIG. 1. 

Detailed Description Text (12) : 

The re-encoding target picture selector 102 selects and labels as target pictures 
the smallest number of pictures that must be re-encoded so that the extracted scene 
can be independently reproduced, that is, reproduced without reference to any 
picture not contained within the scene. 

Detailed Description Text (14) : 

Using the stream structure information, the re-encoding target picture type 
determining means 104 then determines the picture type (I-picture, P-picture, B- 
picture) of the target picture after it is re-encoded . 

Detailed Description Text (16) : 

Based on the stream structure data and the buffer fullness analysis, buffer 
fullness calculating means 106 calculates the buffer fullness at the boundary 
between target pictures and recycled pictures not needing re-encoding . 

Detailed Description Text (17) : 

Re-encoding range code allocation calculator 107 calculates the amount of code to 
allocate to the re-encoding range based on the buffer fullness and number of target 
pictures. The re-encoding range as used herein is a range of pictures bracketing 
the edit position between scenes, specifically containing one or a plurality of 
target pictures at the end of the scene temporally preceding the edit position, and 
one or a plurality of target pictures at the beginning of the scene following the 
edit position. The re-encoding range code allocation calculator 107 calculates the 
amount of code to allocate to the re-encoding range based on the buffer fullness 
and number of target pictures to be re-encoded where the re-encoding range contains 
one or more target pictures in proximity to the point where the two scenes are 
j oined. 

Detailed Description Text (18) : 

The code allocation verifying means 108 verifies whether code allocation T is 
appropriate in terms of amount based on the picture type of the target pictures 
after re-encoding and the code allocation for the re-encoding range. 

Detailed Description Text (19) : 

Re-encoding range expanding means 109 expands the re-encoding range when code 
allocation T is not appropriate. 

Detailed Description Text (20) : 

Re-encoding target picture importance calculating means 110 determines the 
importance of each target picture in the re-encoding range. 

Detailed Description Text (21) : 

Re-encoding target picture target code size calculating means 111 calculates the 
amount of code to allocate to each target picture based on the calculated 
importance rating of each target picture, the code allocation for the re-encoding 
range, the number of target pictures, and the picture type after re-encoding . Note 
that target picture importance is not always needed for this calculation. 
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Detailed Description Text (22) : 

Picture re-encodinq means 112 re-encodes the target pictures based on the 
calculated target picture code size and picture type after re-encoding . 

Detailed Description Text (27) : 

The first step (step 1) is inputting the scene information, which is accomplished 
by the operator using the scene information input means 101 to select the desire d 
scene, that is, a plurality of consecutive frames to be extracted from the original 
bit stream for editing. The re-encoding target picture selector 102 then determines 
the target pictures by detecting t he smallest number of target pictures that must 
be re-encoded in order for the extracted scene to be independently reproducible 
(step 2) . 

Detailed Description Text (28) : 

The stream structure data generator 103 then produces information about the 
structure of the bitstream in the range containing the target pictures and any 
intra-coded pictures referenced for coding the target pictures (step 3), and the 
re-encoding target picture type determining means 104 then determines the picture 
type of the target picture (s) after they are re-encoded (step 4). 

Detailed Description Text (30) : 

The buffer fullness calculating means 106 calculates the buffer fullness at the 
boundary between target pictures and recycled pictures not needing re-encoding 
(step 7), and re-encoding range code allocation calculator 107 calculates the 
amount of code to allocate to the re-encoding range based on the buffer fullness 
and number of target pictures (step 8) . 

Detailed Description Text (31) : 

The code allocation verifying means 108 verifies whether code allocation T is 
appropriate in terms of amount based on the picture type of the target pictures 
after re-encoding and the code allocation for the re-encoding range (that is, 
whether VBV_tail is sufficiently greater than VBV_head as further described below) 
(step 9) . If the code allocation T is not appropriate, the re-encodinq range is 
extended (step 10), and the procedure loops back to step 4. 

Detailed Description Text (32) : 

If it is appropriate, re-encoding target picture importance calculating means 110 
determines the importance of each target picture in the re-encoding range (step 
11) . The re-encoding target picture target code size calculating means 111 then 
calculates the code allocation for each target picture based on the detected 
importance of each target picture, the code allocation for the re-encodinq range, 
the number of target pictures, and the picture type after re-encodinq (step 12) . 

Detailed Description Text (33) : 

The picture re-encoding means 112 then re-encodes the target pictures based on the 
calculated target picture code size and picture type after re-encodinq (step 13), 
and finally scene linking means 113 links the scenes to produce a new bitstream 
(step 14) . 

Detailed Description Text (40) : 
Re-encoding Target Picture Selector 102 

Detailed Description Text (41) : 

In step 2 the re-encodinq target picture selector 102 determines the target 
pictures by detecting the smallest number of pictures that must be re-encoded in 
order for the extracted scene to be independently reproducible. It also identifies 
the range of frames containing the recycled pictures, that is, the range of 
pictures that is encoded in the new bitstream using the original data; this is 
referred to below as the recycled picture range. The identified target pictures are 
re-encoded in step 13, and the recycled picture range encoded in the new bitstream 
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Detailed Description Text (47) : 

Pictures are shown on the top row in FIG. 4 in the coding order, and on the bottom 
row in display order. The first extracted pictures in FIG. 4 are indicated by 
indices SI to S12. The re-encoding target picture, the picture count, and the first 
picture in the recycled picture range selected for pictures SI to S12 are shown in 
the following table. 

Detailed Description Text (48) : 

As shown in Table 1, if the first extracted picture is picture S5 in FIG. 4, the 
corresponding target pictures are P4, B5, and B6, and the first picture in the 
recycled picture range is P7 . After editing, pictures P4, B5, and B6 are re- 
encoded, and picture P7 and later in the coding order are recycled with the 
original picture data. After re-encoding, picture P4 is re-encoded to an I-picture, 
for example, and pictures B5 and B6 are dropped because they are before picture P4 
in the display order. 

Detailed Description Text (54) : 

Pictures are shown on the top row in FIG. 5 in the coding order, and on the bottom 
row in display order. If the last extracted pictures are indicated by indices E4 to 
E12 in FIG. 5, the re-encoding target picture, the picture count, and the last 
picture in the recycled picture range are as shown in the following table. 

Detailed Description Text (55) : 

As shown in Table 2, the target picture for last extracted picture E5 is B5, and 
the picture B3 is the end of the recycled picture range. After editing, picture B5 
is re-encoded to an I-picture or P-picture, and picture P4 is dropped because it 
temporally follows picture B5 . Pictures B3 and before in the coding order are 
within the recycled picture range and are therefore simply recycled using the 
original picture data. 

Detailed Description Text (56) : 

The basic method for target picture selection is described above. Target pictures 
for re-encoding are thus identified for each scene at the beginning of the scene 
and at the end of the scene. The recycled picture range in each scene further 
extends from the first picture in the recycled picture range to the last picture in 
the recycled picture range. By thus re-encoding only the specifically selected 
target pictures, each scene can be reproduced independently of any pictures not 
contained in the scene frame sequence. Re-encoding is thus limited to the smallest 
number of pictures possible, and processing can therefore be efficiently 
accomplished. 

Detailed Description Text (59) : 

This stream structure information includes the byte location of each picture, and 
the header information added to each picture, and includes the correlated I-picture 
required for re-encoding the target pictures. 

Detailed Description Text (64): 

Re-encoding Target Picture Type Determining Means 104 
Detailed Description Text (65) : 

In step 4, the re-encoding target picture type determining means 104 determines the 
picture type (I-picture, P-picture, B-picture) of the target picture after it is 
re-encoded . The picture type after re-encoding is determined as follows in the 
present embodiment. 

Detailed Description Text (66) : 

If the target picture is the first extracted picture, it is an I-picture. A picture 
temporally preceding the target picture is a P-picture. The target pictures are 
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then re-encoded in step 13 to produce the picture types determined in step 4 . 
Detailed Description Text (67) : 

It should be noted that while this preferred embodiment of the invention uses 
either I-pictures or P-pictures as the picture type after re-encoding, B-pictures 
can also be obviously used. It is also possible to use only I-pictures. It is, 
however, preferable to select the picture types resulting in the smallest possible 
code so as to prevent great change in the stream. 

Detailed Description Text (70) : 

In step 6, buffer fullness analyzing means 105 analyzes the change in buffer 
fullness. This step is only performed when the edit stream is VBR coded. In this 
case it is not possible to calculate the amount of code to allocate to the re- 
encoding range using the equation (equation 6) for calculating the re-encoding 
range code allocation. This is because vbv_delay is used to calculate the VBV_tail 
(buffer fullness immediately before the next picture after (in the coding order) 
the last picture in the recycled picture range of the scene) and VBV_head (buffer 
fullness immediately before the first picture in the recycled picture range of the 
scene) values used in equation 6, and vbv_delay is always OxFFFF in VBR coded data. 



Detailed Description Text (76) : 

In step 7 the buffer fullness calculating means 106 calculates the buffer fullness 
at the boundary between target pictures and recycled pictures not needing re- 
encoding when the edit stream is CBR coded. This boundary occurs in each scene at 
the first picture in the recycled picture range and the next picture after the last 
picture in the recycled picture range. 

Detailed Description Text (85) : 

Re-encoding Range Code Allocation Calculator 107 
Detailed Description Text (86) : 

In step 8, re-encoding range code allocation calculator 107 calculates the amount 
of code T to allocate to the re-encoding range ( re-encoding range code allocation 
T) . This re-encoding range code allocation T is calculated by applying buffer 
fullness VBV_head and VBV_tail calculated in steps 6 and 7 to equation (6) . 
##EQU6## 

Detailed Description Text (90) : 

VBV_delay_head is vbv_delay of first picture in the recycled picture range after 
the edit point (at the beginning of the scene). R is a bit rate. N is a number of 
re-encoding target pictures at the edit point. F is a frame rate. 

Detailed Description Text (92) : 

As shown in FIG. 6(a), if the last extracted picture is picture B6 in scene 1, the 
target pictures are pictures B5 and B6 and the recycled picture range from picture 
B3 and before. If the first extracted picture in scene 2 is B25 as shown in FIG. 6 
(b) , the target pictures are P24, B25, and B26, and the recycled picture range is 
from picture P27 and after. When scenes 1 and 2 and joined, target pictures B5, B6, 
B25, B26, and P24 are re-encoded to P5 ' , P6 1 , 125', P26 1 , and P24'. 

Detailed Description Text (93) : 

To prevent buffer underflow and buffer overflow, it is only necessary to ensure (a) 
that buffer fullness VBV_tail at the decoding time of the last picture in the 
recycled picture range B3 (that is, immediately before picture P4 (in the coding 
order) after picture B3 is decoded) is the same before and after the scenes are 
joined, and (b) that buffer fullness VBV-head at decoding time of the first picture 
in the recycled picture range P27 is also the same before and after the scenes are 
joined. Accomplishing this by allocating the five re-encoded pictures (P5 1 , P6 1 , 
125 f , P26 ! , P24*) to the range in which the target pictures are allocated (picture 
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B3 decoding time to picture P27 decoding time) is considered next. 
Detailed Description Text (94) : 

From FIG. 6 we know that the code T allocated to this range is equal to the sum of 
the difference of the code size at the decoding time of the next picture after 
picture B3 and the code size at the decoding time of picture P27 (VBV_tail — 
VBV head) to the code input during the time of the five re-encoded pictures 
(R. times .N/F) . 

Detailed Description Text (96) : 

Re-encoding range code allocation T is thus obtained by calculating VBV_tail and 
VBV_head and applying these values to equation (6) . Note that if there is no scene 
before the edit point, VBV_tail=VBV buffer size; if there is no scene after the 
edit point, VBV_head=0. 

Detailed Description Text (98) : 

In step 9 the code allocation verifying means 108 verifies whether code allocation 
T is appropriate based on the picture type of the target pictures after re-encoding 
and the. If code allocation T is appropriate, the procedure skips to step 11; 
otherwise control goes to step 10. 

Detailed Description Text (100) : 

Next, the initial total code allocation for the re-encoding range is calculated 
from the number of target pictures in the re-encoding range and the picture types 
after re-encoding . This initial, code total TO can be calculated from the following 
equation where Ni is the number of I-pictures, Np is the number of P-pictures, and 
Nb is the number of B-pictures, after re-encoding . 

Detailed Description Text (103) : 

This maximum compression ratio is determined by evaluating whether the picture 
deterioration after re-encoding is acceptable, and is a value from 0 to 1 . From 
experience, we have found a maximum compression ratio of 0.6 preferable. 

Detailed Description Text (104) : 

The code allocation verifying means 108 determines the code size to be 
inappropriate in step 9 even when there is no target picture at the edit point. 
This is because buffer control is accomplished in this exemplary embodiment by 
adjusting the code size of the re-encoded pictures. If there is no target picture, 
buffer control is difficult. 

Detailed Description Text (105) : 
Re-encoding Range Expanding Means 109 

Detailed Description Text (106) : 

If the code allocation T is determined not appropriate in step 9, re-encoding range 
expanding means 109 extends the re-encoding range in step 10, and the procedure 
then loops back to step 4. 

Detailed Description Text (107) : 

Extending the re-encoding range means increasing the number of target pictures for 
re-encoding . If in step 10 there is no re-encoding target picture at the edit 
point, a target picture is selected so that there is at least one re-encoded 
picture. Code allocation T for the target picture in the code allocation equation 
(6) depends on VBV_tail--VBV_head. The re-encoding range is therefore expanded to 
increase VBV_tail--VBV_head . 

Detailed Description Text (108) : 

More specifically, re-encoding range expanding means 109 searches in the direction 
that will extend the re-encoding range, and re-encodes the first intra-coded 
picture (a I-picture) in the recycled picture range to a forward predictive-coded 
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picture {a P-picture) . The re-encodinq range expanding means 109 is described in 
further detail with reference to FIG. 4, FIG. 5, FIG. 7, and FIG. 8. 



Detailed Description Text (109) : 

Operation when processing the beginning of the scene is described first. FIG. 7 
illustrates the process for extending the re-encoding range when the first 
extracted picture is S5 in FIG. 4. FIG. 7(a) shows the re-encoding target pictures 
after step 2. FIG. 7(b) shows the result of re-encoding with the target pictures 
selected in FIG. 7(a). FIG. 7(c) shows the re-encodinq target pictures after step 
10. FIG. 7(d) shows the result of re-encoding with the target pictures selected in 
FIG. 7 (c) . 

Detailed Description Text (111) : 

After editing pictures P4, B5, and B6 are re-encoded to P4', 15 f , and P6 1 as shown 
in FIG. 7(b) . Furthermore, the original picture data is used for pictures P7 and 
after in the coding order. The order of the pictures in the coding order is 15', 
P6', P4*, and P7 and after. 

Detailed Description Text (112) : 

Note that the number of target pictures is increased in this case as shown in FIG. 
7(c). Using the stream structure data, the first I-picture (that is, picture 110) 
found searching forward from the first picture in the recycled picture range 
(referring to FIG. 7(a), i.e., picture P7) is designated the last target picture in 
the re-encoding range. This means that the target picture range is extended from 
pictures P4, B5, and B6 to include pictures P4, B5, B6, P7, B8, B9, and 110. 

Detailed Description Text (113) : 

After editing, pictures P4, B5, B6, P7, B8, B9, and 110 are re-encoded as pictures 
P4 1 , 15', P6', P7', B8 1 , B9 f , and P10 1 . The first picture in the recycled picture 
range is also updated to picture Bll. During re-encoding, the last I-picture in the 
range of re-encoding target pictures is changed to a P-picture, thereby reducing 
the code size of the picture. 

Detailed Description Text (114) : 

By thus extending the re-encodinq range and re-encodinq I-picture 110 to a P- 
picture P10 1 , VBV_head can be reduced. VBV_tail — VBVJiead is thus increased, and 
the code allocation for each picture is thereby increased. 

Detailed Description Text (115) : 

FIG. 8 shows the change in buffer fullness before and after this process. The code 
size allocated to pictures 15', P6', and P4 1 is considered in FIG. 8. If VBV_head 
is the code size immediately before decoding picture P7 before extending the~~ re- 
encodinq range, VBV_head' is the code size at the corresponding time after 
extending the range. Note that VBV_head' is less than VBV_head. 

Detailed Description Text (116) : 

The code allocation for pictures 15 f , P6 ' and P4 * is greater after this range 
extending process. This is because an I-picture, which consumes a large amount of 
code, is re-encoded as a P-picture, which uses less code compared with an I- 
picture . 

Detailed Description Text (117) : 

The process extending the re-encodinq range at the end of the scene is described 
next. FIG. 9 illustrates the process for extending the re-encodinq range when the 
last extracted picture is E5 in FIG. 5. FIG. 9(a) shows the re-encodinq target 
pictures after step 2. FIG. 9{b) shows the result of re-encodinq with the target 
pictures selected in FIG. 9(a). FIG. 9 (c) shows the re-encodinq target pictures 
after step 10. FIG. 9(d) shows the result of re-encodinq with the target pictures 
selected in FIG. 9(c). 
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Detailed Description Text (119) : 

After editing, picture B5 is re-encoded to P5 1 as shown in FIG. 9(b). Furthermore, 
the original picture data is used for pictures B3 and before in the coding order. 
The order of the pictures in the coding order is up through B3, and then P5 ' . 

Detailed Description Text (120) : 

Note that the number of target pictures is increased in this case as shown in FIG. 
9(c). The first target picture in the re-encoding range is reset to the first I- 
picture (that is, picture II) found searching backward from the last picture in the 
recycled picture range (referring to FIG. 9(a), i.e., picture B3) using the stream 
structure data. This means that the target picture range is extended from picture 
B5 to pictures II, B2, B3, and B5 . In this case, P-picture P4 is reproduced outside 
the extracted scene range, and is thus deleted. 

Detailed Description Text (121) : 

After editing, pictures II, B2, B3, and B5 are re-encoded to pictures PI ' , B2 1 , 
B3 ' , and P5 1 . The last picture in the recycled picture range is also updated to the 
picture immediately before picture II. By thus re-encoding the first I-picture in 
the re-encoding range to a P-picture, the code size of that picture is reduced. 

Detailed Description Text (122) : 

By thus extending the re-encoding range and re-encoding I-p.icture _I1 to a P-picture 
PI 1 , VBV_tail can be increased. VBV_tail--VBV_head is thus increased, and the code 
allocation for each picture is thereby increased. 

Detailed Description Text (123) : 

FIG. 10 shows the change in buffer fullness before and after this process. The code 
size allocated to pictures P5 1 is considered in FIG. 10. If VBV_tail is the code 
size immediately before decoding the next picture after picture B3 before extending 
the re-encoding range, VBV_tail' is the code size at the corresponding time after 
extending the range. Note that VBV^ail 1 is greater than VBV_tail. 

Detailed Description Text (124) : 

The code allocation for picture P5 ' is greater after this range extending process. 
This is because an I-picture, which consumes a large amount of code, is re-encoded 
as a P-picture, which uses less code compared with an I-picture. 

Detailed Description Text (126) : 

Re-encoding Target Picture Importance Calculating Means 110 
Detailed Description Text (127) : 

In step 11, re-encoding target picture importance calculating means 110 determines 
the importance of each target picture in the re-encoding range. This process is 
described more fully below with reference to FIG. 11 FIG. 11 shows the process for 
calculating target picture importance for scene 1 and scene 2 extracted from edit 
stream sources 1 and 2, respectively, and then combined to a new video stream. 

Detailed Description Text (128) : 

Pictures are shown in the display order in FIG. 11. Step 2 identifies pictures B12 
and B13 in scene 1, and pictures B2 to P7 in scene 2, as the target pictures at the 
edit point between scenes 1 and 2. Step 4 then specifies the picture type after re- 
encoding as indicated from picture P12 1 to P8 1 . This consecutive sequence of target 
pictures is the re-encoding range. 

Detailed Description Text (129) : 

The importance rating of each target picture in the re-encoding range is initially 
set uniformly to the same value. In this exemplary embodiment of the invention the 
picture importance rating is initially set at 50 as shown in FIG. 11(a). 

Detailed Description Text (131) : 
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The re-encoding ranges are considered separately in each scene. At the beginning of 
a scene, a constant B multiplied by the number of target pictures in the re- 
encoding range starting with the second target picture is added to the importance 
rating of the first target picture, that is, the number of pictures in the re- 
encoding range minus 1. This is shown in FIG. 11(b). The beginning of scene 2 is I- 
picture 12'. Six target pictures follow picture 12'. Constant B in this exemplary 
embodiment is 1. Therefore { 6. times. 1) is added to the initial importance rating of 
picture 12', which is 50 now, and the importance of picture 12' is thus reset to 
56. The importance of the first picture in a scene is thus increased. 

Detailed Description Text (134) : 

This is because the edit point is the point at which the scenes change, and 
pictures in proximity to the scene change are difficult for the viewer to perceive. 



Detailed Description Text (135) : 

Target picture importance is also adjusted up or down by comparing the code size of 
the target pictures before editing with the average code size of each picture based 
on the picture type of the target pictures before editing and the picture type 
after re-encoding . 

Detailed Description Text (137) : 

where Dorg is an original code size of the re-encoding target picture. 
Detailed Description Text (140) : 

If original code size Dorg of the original picture B12 is 1% smaller than average 
code size Dave for B type pictures, the importance of the corresponding re-encoded 
picture P12 1 after re-encoding picture B12 is reduced by subtracting 1 from 50, 
obtaining an importance rating of 49. 

Detailed Description Text (141) : 

Furthermore, if original code size Dorg of the original picture B13 is 1% smaller 
than average code size Dave for B type pictures, the importance of the 
corresponding re-encoded picture P13' after re-encoding picture B12 is reduced by 
subtracting 1 from 49, obtaining an importance rating of 48. 

Detailed Description Text (142) : 

Furthermore, if original code size Dorg of the original picture B2 is 2% greater 
than average code size Dave for B type pictures, the importance of the 
corresponding re-encoded picture 12 ' after re-encoding picture B2 is increased by 
adding 2 to 56, obtaining an importance rating of 58. 

Detailed Description Text (143) : 

Furthermore, if original code size Dorg of the original picture B4 is 1% greater 
than average code size Dave for B type pictures, the importance of the 
corresponding re-encoded picture P4 1 after re-encoding picture B4 is increased by 
adding 1 to 50, obtaining an importance rating of 51. 

Detailed Description Text (144) : 

If original code size Dorg of the original picture P8 is 5% greater than average 
code size Dave for P-type pictures, the importance of the corresponding re-encoded 
picture P8 1 after re-encoding picture P8 is increased by adding 5 to 50, obtaining 
an importance rating of 55. 

Detailed Description Text (146) : 

Re-encoding Target Picture Target Code Size Calculating Means 111 
Detailed Description Text (147) : 

In step 12, the re-encoding target picture target code size calculating means 111 
calculates the code allocation (target code size) for each target picture based on 
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the importance of each target picture, the re-encoding range code allocation, the 
number of target pictures, and the picture type after re-encoding . 

Detailed Description Text (151) : 

First, the average importance of the target pictures contained in the re-encoding 
range is obtained. Next, (target picture importance rating) /( average importance) is 
multiplied by the target code size for each picture to increase or decrease the 
target code size. Let us assume that the total target code size of each target 
picture at this point is Tl. 

Detailed Description Text (152): 

Next, the difference between the re-encoding range code allocation and the target 
code size is distributed to each of the target pictures according to the ratio of 
the average code size of the pictures. Finally, it is verified that buffer 
underflow and buffer overflow will not occur at the target picture decoding timing; 
if either a buffer underflow or overflow will occur, the target code size of each 
target picture is adjusted again. This operation is further described below. 

Detailed Description Text (153) : 

First, the difference Tdiff between re-encoding range code allocation T and total 
target code size Tl calculated in the previous process is calculated using the 
following equation. 

Detailed Description Text (155) : 

Code distribution Di_add, Dp_add, and Db_add distributed to the target pictures 
according to picture type is calculated using the following equations (10) 
according to the picture type after re-encoding the target pictures. ##EQU10## 

Detailed Description Text (156) : 

where Ni is a number of re-encoding target pictures in the re-encoding range that 
are I-pictures after re-encoding . 

Detailed Description Text (157) : 

Np is a number of re-encoding target pictures in the re-encoding range that are P- 
pictures after re-encoding . 

Detailed Description Text (158) : 

Nb is a number of re-encoding target pictures in the re-encoding range that are B- 
pictures after re-encoding . 

Detailed Description Text (161) : 

Db is an average code size of B-pictures. Note that Di, Dp, and Db in equation (10) 
are the average code size values calculated for each picture type in step 9. Target 
code size values Di_targ, Dp_targ, and Db_targ are then calculated using equation 

(11) for picture types I, P, and B, respectively, after re-encoding . 

Detailed Description Text (163) : 

Finally, whether a buffer overflow or buffer underflow will occur at the target 
picture decode timing is verified, and if either will, the target code size of each 
target picture is again adjusted. This verification is performed using equation 

(12) below from the first target picture in the re-encoding range near the edit 
point in sequence to the last target picture. ##EQU11## 

Detailed Description Text (167) : 

n is the picture sequence of target pictures in the re-encoding range where (n=l, . 
. . Nr) . n=l: first target picture n=Nr: last target picture R is a bit rate. F: 
frame rate VBV_SIZE is a VBV buffer size. Psize(n) is a target code size of picture 



n . 



Detailed Description Text (170) : 
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The total code size of the re-encoded pictures thus becomes equal to the target 
total code size. The target code size for each target picture is thus determined. 

Detailed Description Text (171) : 
Picture Re-encoding Means 112 

Detailed Description Text (172) : 

In step 13, the picture re-encoding means 112 re-encodes the target pictures based 
on the calculated target picture code size and the picture type after re-encoding . 
The I-picture referenced for coding the first target picture is first found in the 
stream structure data, and is decoded. 

Detailed Description Text (173) : 

The re-encoder continues searching forward in the stream to decode the I-pictures, 
P-pictures, and target pictures for re-encoding . Future and past referenced 
pictures, that is, pictures used for predictive coding P-pictures or B-pictures, 
are buffered to temporary storage at this time. 

Detailed Description Text (174) : 

For example, a P-picture is decoded using the first I-picture or P-picture 
encountered in the stream as a "future referenced picture." When a re-encoding 
target picture is found, the picture is decoded and then re-encoded to match the 
target code size. 

Detailed Description Text (175) : 

If the target picture cannot be coded to match the target code size, the code size 
is adjusting by adding invalid data such as stuffing or padding bytes, or the 
difference between the target code size and the coded size of the target picture is 
added to the target code size of the next target picture. If the picture type after 
re-encoding is a P-picture, the picture is buffered to memory for use as a future 
referenced picture. Processing ends when all target pictures have been re-encoded . 

Detailed Description Text (178) : 

If three scenes are to be linked in the sequence scene 1, scene 2, and scene 3, the 
target pictures in each of the scenes are re-encoded by the process through step 13 
so that each scene can be independently reproduced, and the scenes are extracted 
from the original bitstream. The scenes at this point are shown in FIG. 12(a). In 
step 14, the scenes thus prepared are connected in the specified sequence as shown 
in FIG. 12(b). This is accomplished as described below. 

Detailed Description Text (179) : 

First, the re-encoded target pictures at the beginning of scene 1 are connected in 
coding order. If the picture type of the target picture before editing is different 
from the picture type after re-encoding, the pictures are rearranged based on the 
display order in a coding order conforming to the picture type after target picture 
re-encoding . 

Detailed Description Text (180) : 

Next, the re-encoded target pictures at the beginning of the scene are connected to 
the recycled picture range of scene 1. The re-encoded target pictures at the end of 
scene 1 are then connected in coding order. 

Detailed Description Text (189) : 

The re-encoding target picture type determining means of the present invention 
extensively uses picture types with a code size smaller than I-pictures, allocates 
efficiently, and suppresses picture quality deterioration. 

Detailed Description Text (190) : 

The code allocation verifying means and re-encoding range expanding means of the 
present invention extend the range of re-encoded frames when necessary, verify that 
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the allocated code is always within an appropriate range, and thus maintain the 
overall quality of the video. 

Detailed Description Text (191) : 

B Y re-encoding from an I-picture to a P-picture, the re-encoding range expanding 
means of the invention increases the code allocation to other pictures, and thus^"/[\ 
achieves even better picture quality. 

Detailed Description Text (192) : 

The re-encoding target picture importance calculating means assures that more code 
is allocated to pictures with a high importance rating, less is allocated to 
pictures of lesser importance, and thus assures efficient code allocation. This 
further prevents propagating picture quality loss at the edit point, and thus 
maintains the overall quality of the video. 

Detailed Description Paragraph Table { 1 ) : 

TABLE 1 First No. of First picture in extracted re-encoded Re-encoded recycled 
picture picture pictures pictures range SI 1 II P4 S2 3 II, B2, B3 P4 S3 2 II, B3 
P4 S4 1 P4 P7 S5 3 P4, B5, B6 P7 S6 2 P4, B6 P7 S7 1 P7 110 S8 3 P7, B8, B9 110 S9 
2 P7, B9 110 S10 1 110 P13 Sll 3 110, Bll, B12 P13 S12 2 110, B12 P13 

Detailed Description Paragraph Table (2) : 

TABLE 2 Last No. of Last picture in extracted re-encoded Re-encoded recycled 
picture picture pictures picture range E4 0 B6 E5 1 B5 B3 E6 2 B5, B6 B3 E7 0 B9 E8 
1 B8 B6 E9 2 B8, B9 B6 E10 0 B12 Ell 1 Bll B9 E12 2 Bll, B12 B9 

CLAIMS : 

1. A video editing apparatus comprising: a scene information input means for 
inputting scene information, where a scene is a plurality of consecutive frames 
extracted from an edit stream; a re-encoding target picture selector for selecting 
as target pictures for re-encoding the smallest number of pictures that must be re- 
encoded for the scene to be independently reproducible; a stream structure data 
generator for generating structure information for a stream in a range containing a 
target picture and an intra-coded picture referenced for coding the target picture; 
a buffer fullness calculating means for calculating, from stream structure data, 
buffer fullness at a target picture boundary to a recycled picture not requiring 
re-encoding ; a re-encoding range code allocation calculator for calculating a code 
allocation to a re-encoding range based on buffer fullness and target picture 
count, said re-encoding range being one or a plurality of target pictures near an 
edit point between scenes; a re-encoding target picture target code size 
calculating means for calculating a code allocation to each target picture based on 
the re-encoding range code allocation, target picture count, and picture type after 
re-encoding ; and a scene linking means for connecting scenes and producing a new 
stream. 

2. A video editing apparatus as described in claim 1, further comprising: a re- 
encoding target picture type determining means for deciding, from the stream 
structure data, a target picture type after re-encoding ; and a picture re-encoding 
means for target picture re-encoding based on target picture target code size and 
picture type after re-encoding . 

3. A video editing apparatus as described in claim 2, further comprising: a code 
allocation verifying means for verifying whether the code allocation is appropriate 
based on the re-encoding range code allocation and target picture picture type 
after re-encoding ; and a re-encoding range expanding means for extending the re- 
encoding range if the code allocation is not appropriate. 

4. A video editing apparatus as described in claim 3, wherein the re-encoding range 
expanding means searches in a direction extending the re-encoding range, and re- 
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encodes a first intra-coded picture found in the recycled picture range to a 
forwardly predictive-coded picture. 

5. A video editing apparatus as described in claim 1, further comprising a re- 
encoding target picture importance calculating means for calculating an importance 
rating for each target picture in the re-encoding range. 

7. A video editing method, comprising: a scene information input step for inputting 
scene information, where a scene is a plurality of consecutive frames extracted 
from an edit stream; a re-encoding target picture step for selecting as target 
pictures for re-encoding the smallest number of pictures that must be re-encoded 
for the scene to be independently reproducible; a stream structure data generating 
step for generating structure information for a stream in a range containing a 
target picture and an intra-coded picture referenced for coding the target picture; 
a buffer fullness calculating step for calculating, from stream structure data, 
buffer fullness at a target picture boundary to a recycled picture not requiring 
re-encoding ; a re-encoding range code allocation calculating step for calculating a 
code allocation to a re-encoding range based on buffer fullness and target picture 
count, said re-encoding range being one or a plurality of target pictures near an 
edit point between scenes; a re-encoding target picture target code size 
calculating step for calculating a code allocation to each target picture based on 
the re-encoding range code allocation, target picture count, and picture type after 
re-encoding ; and a scene linking step for connecting scenes and producing a new 
stream. 



8. A video editing method as described in claim 7, further comprising: a re- 
encoding target picture type determining step for deciding, from stream structure 
data, a target picture picture type after re-encoding ; and a picture re-encoding 
step for target picture re-encoding based on target picture target code size and 
picture type after re-encoding . 

9. A video editing method as described in claim 8, further comprising: a code 
allocation verifying step for verifying whether the code allocation is appropriate 
based on the re-encoding range code allocation and target picture picture type 
after re-encoding ; and a re-encoding range expanding step for extending the re- 
encoding range if the code allocation is not appropriate. 

10. A video editing method as described in claim 9, wherein the re-encoding range 
expanding step searches in a direction extending the re-encoding range, and re- 
encodes a first intra-coded picture found in the recycled picture range to a 
forwardly predictive-coded picture. 

11. A video editing method as described in claim 7, further comprising a re- 
encoding target picture importance calculating step for calculating an importance 
rating for each target picture in the re-encoding range. 

13. A data storage medium having a computer readable instructions recorded thereon, 
the computer readable instructions including instructions capable of instructing a 
computer to: input scene information, where a scene is a plurality of consecutive 
frames extracted from an edit stream; select, as target pictures for re-encoding, 
the smallest number of pictures that must be re-encoded for the scene to be 
independently reproducible; generate structure information for a stream in a range 
containing a target picture and an intra-coded picture referenced for coding the 
target picture; calculate, from stream structure data, buffer fullness at a target 
picture boundary to a recycled picture not requiring re-encoding ; calculate a code 
allocation to a re-encoding range based on buffer fullness and target picture 
count, said re-encoding range being one or a plurality of target pictures near an 
edit point between scenes; calculate a code allocation to each target picture based 
on the re-encoding range code allocation, target picture count, and picture type 
after re-encoding ; and connect scenes and produce a new stream. 
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14. A data storage medium as described in claim 13, wherein the computer readable 
instructions further include instructions operable to instruct the computer to: 
decide, from stream structure data, a target picture picture type after re- 
encoding ; and re-encode a target based on target picture target code size and 
picture type after re-encoding . 

15. A data storage medium as described in claim 14, wherein the computer readable 
instructions further include instructions operable to instruct the computer to: 
verify whether the code allocation is appropriate based on the re-encoding range 
code allocation and target picture type after re-encoding ; and extend the re- 
encoding range if the code allocation is not appropriate. 

16. A data storage medium as described in claim 15, wherein the computer readable 
instructions capable of instructing the computer to extend the re-encoding range 
comprise instructions operable to instruct the computer to search in a direction 
extending the re-encoding range, and re-encode a first intra-coded picture found in 
the recycled picture range to a forwardly predictive-coded picture. 

17. A data storage medium as described in claim 13, further comprising computer 
readable instructions operable to instruct the computer to calculate step for 
calculating an importance rating for each target picture in the re-encoding range. 
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